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The common purposes of the studies conducted in language program evaluations 
are to examine the match between what is desired for the programme versus the 
actual state of the programme, to make judgments about learners ’ level of skills 
and knowledge, and to make suggestions for improvement. However, it is not 
currently common practice in Turkey either to develop language teaching 
programmes based on the Common European Framework as a reference, or to 
introduce improvements in these programmes based on an evaluation of their 
effectiveness. This study aims to describe the process of developing a new teaching 
programme, taking CEF into consideration, at the Preparatory’ Programme at the 
School of Foreign Languages, Izmir University of Economics, and also to evaluate 
the effectiveness of the programme. 236 Freshman students and 48 faculty 
members from 5 different faculties participated in the study. The results indicated 
a significant relationship between students ’ proficiency scores and perception of 
their own competencies and a significant difference in perception of their own 
competence in terms of levels at the preparatory program. Although faculty 
members stated that Preparatory’ Program, in general, meets the needs of the 
students, students still have difficulty in practising some tasks requiring higher 
order thinking skills. The study suggests a series of learner training sessions to 
raise the awareness of the students, extending duration of the modules, reviewing 
the order of objectives in Intermediate and Upper-Intermediate, and working in 
cooperation with Faculties in order to increase awareness of mutual expectations. 

Key Words: language program evaluation, curriculum development, language learning, 
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INTRODUCTION 

The idea of curriculum is not new, however, the way we understand and 
theorize it has changed over the years. Omstein and Hunkins (2004) define 
curriculum as a plan that includes strategies for achieving desired goals. Much 
earlier, Bobbitt (1923) and Taba (1962) had approached curriculum from a 
wider perspective, describing the curriculum as a plan for learning, a production 
system, and a programme evaluation of the outcomes. Programme evaluation, 
which is a very significant part of curriculum, has also been defined differently 
in literature. While some educators relate evaluation with measurement, the 
others define it as the assessment of specific objectives which need to be 
attained (Worthen and Sanders, 1988). Tyler (1991) briefly summarizes 
programme evaluation as a process essential to curriculum development. As 
indicated in all these definitions, programme evaluation is seen as a systematic 
effort that includes the collection, analysis and synthesis of information. 

Programme evaluation can be conducted for two main purposes: firstly, 

improving the programme, known as formative evaluation, and secondly, 
deciding whether a programme should be continued, called summative 
evaluation. Formative evaluation is conducted to collect and share information 
for programme improvement, through which problematic aspects are identified 
and possible solutions are worked on (Scriven, 1991). Summative evaluation, 
on the other hand, is conducted at the end of a programme to provide 
information about that programme’s success (Worthen, 1990). Researchers 
emphasize that programmes should be evaluated regularly using either one or 
the other of these methods, and that decisions made should be based on research 
which covers the identification of measurable student learning objectives and 
outcomes, the assessment of the degree to which students are meeting the 
objectives, the evaluation and sharing of results, leading to consequent changes 
to the programme as deemed necessary (Scriven, 1991; Worthen, 1990). 

Programme evaluations have been conducted in every single subject, discipline 
and field, including English Language Teaching as a Foreign Language. There 
has been an increasing need for developing and evaluating new language 
teaching programmes, in particular, after the introduction of intensive foreign 
language instruction into the Turkish education system. In the last two decades, 
programme evaluations of the Preparatory programmes at universities have 
become increasingly important. These evaluations are designed to ensure that, 
during the course of one academic year, students reach a proficient level of 
English, which will enable them to follow their courses effectively. Because of 
this mission attributed to preparatory programmes/schools, it is essential that the 
preparatory school programmes be evaluated regularly so that their strengths 
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and weaknesses can be identified and goals can be achieved. This need has led 
to a number of studies. In the study conducted by Gullu (2007), the English 
programme at Vocational School of Q’ukurova University was evaluated from 
the students’ point of view, revealing certain problems such as the difficulty of 
the course content, course materials that appeared unattractive and lacked 
relevance, lack of motivation and interest, and inappropriate physical 
conditions. The data revealed that the programme did not match students’ 
expectations and needs, and therefore, was in urgent need of reviewing, 
revision, and improvement. Erdem (1999) investigated the effectiveness of 
English language curriculum at Middle East Technical University and found 
that the program was overly teacher-centered and recommended its replacement 
with a more student-centered one. In addition, the study also revealed a need to 
improve in-service training and to set up an ongoing curriculum evaluation 
system. A similar study by Gerede (2005) at Anadolu University compared the 
old and renewed curricula of Preparatory Programme and the results showed 
that a significant improvement had occurred in terms of meeting the students’ 
language needs. A study by Topgu (2005) examining the implementation of the 
theme-based curriculum and objectives of the Basic Education Department at 
the Middle East Technical University indicated a great difference between the 
perceptions of teachers and students. Karata§ (2007) evaluated the syllabus of 
English instruction programme at Yildiz Teknik University by collecting data 
from teachers and students, revealing significant differences between the 
teachers’ and students’ opinions on the suitability of the programme’s 
objectives, the audio-visual materials used in the programme, the tasks, 
students’ participation, and the suitability of the activities. The results 
highlighted the negative opinions of the teachers, who emphasized that the 
programme had no positive effect on the students’ listening, speaking or 
grammar knowledge. 

The common purposes of the studies mentioned above are to examine the match 
between what is desired for the programme versus the actual state of the 
programme, to make judgments about learners’ level of skills and knowledge, 
and to make suggestions for improvement. However, it is not currently 
common practice in Turkey either to develop language teaching programmes 
based on the Common European Framework as a reference, or to introduce 
improvements in these programmes based on an evaluation of their 
effectiveness. The Common European Framework (CEF) is a guideline for 
languages and is used as a reference point. It encourages programmes in 
language education to develop skills, create a system of continuous assessment; 
it also presents a basis for the mutual recognition of language qualifications and 
educational mobility. The framework provides a common basis for the explicit 
description of objectives, content and methods, and thus enhances the 
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transparency of courses, syllabus and qualifications (Council of Europe, 2003). 
For this reason, it is considered that designing a new English teaching 
programme based on CEF will lead to a sound and well-grounded syllabus. 
Many Asian and European countries as well as USA and Canada have either 
adopted CEF or made CEF based adaptations to their language teaching 
curricula. In the US, educational frameworks in every state borrowed scales 
from CEF. In Ireland, English Language Proficiency Benchmarks have been 
developed based on CEF for learners of English as a second language in 
primary and post-primary (secondary) schools. The aim is to offer a curriculum 
which can improve the learners’ language so that they can access English- 
medium education without intensive support. Moreover, in China, the 
government is planning to change high school exit assessment and the 
curriculum according to CEF. In such a case, millions of people will be affected 
by this innovation (Little, 2006). Likewise, Korea is planning to implement 
changes in line with CEF in order that Korean students can have acceptable 
language qualifications when entering the global job market (Finch, 2009). 
However, the situation is not similar in Turkey as CEF addresses only general 
English whereas the Preparatory Programmes in English-medium universities in 
Turkey aim to teach General English while at the same time prepare their 
learners for the academic studies in their faculties. Therefore, their syllabi must 
include aspects of both General and Academic English. When this reality is 
taken into consideration in these settings, the implementation of CEF alone 
would be insufficient for the accomplishment of the objectives in the 
Preparatory Programmes, because CEF covers descriptions of objectives in 
General English only, and excludes Academic English. Thus, this study 
describes, in detail, how the Preparatory English Teaching Programme at Izmir 
University of Economics has been developed on the basis of the CEF, but also 
incorporating specifically academic objectives for each level. The study firstly 
gives information about the curriculum development process, importance of the 
study, then methodology, results, discussion, and conclusion. 

Previous System at the Preparatory Programme 

School of Foreign Languages (SFL) runs the Preparatory Programme, which 
prepares the students for their Faculties through an intensive English 
preparatory year. Over the previous eight years of its existence, the school 
followed a semester-based system. An entire academic year in the semester 
based system consisted of two fourteen-week semesters, with no learner 
requirements needed to pass between levels. The three quizzes within the 
semester and one mid-term exam at the end of the semester only served to allow 
calculation of the average grade, which the administration used to decide 
whether the learner could enter the Proficiency in English Exam, the sole 
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requirement for learners to proceed with their undergraduate programmes. A 
questionnaire was given both to faculty members and students to see the 
effectiveness of the system. The feedback received showed inadequacies in 
students’ proficiency in faculty classes, and this led the administration to search 
for new approaches. 

Transition to the new system - “Modular System” 

Having worked on new alternatives for one and a half years in order to develop 
the existing curriculum, Modular System was decided to be adopted for the 
2009-2010 academic year. During the development of this teaching 
programme, the CEF was used as a resource tool, taking into account the needs 
of the setting. With the introduction of the modular system at the Preparatory 
Programme, an urgent need was felt to review and revise the programmes. The 
steps followed in this process are as follows: 

- A needs analysis was conducted with the faculty members and Freshman 
students to identify the main needs of students and the expectations of faculty 
members, which later led the curriculum designers to develop the current 
curriculum accordingly. 

- Having considered the needs of Faculty, the outcome of Preparatory 
Programme was defined as B2/CEF. Taking the need to reach this level as the 
finishing point, the Curriculum Unit accordingly set objectives for each level, 
Beginner (A), Elementary (B), Pre-Intermediate (C), Intermediate (D), and 
Upper Intermediate (E) levels. The identified aims/objectives were converted 
into linguistic requirements - “can do” statements of the CEF-, specifying not 
only the knowledge and skills, but also the ability level that the learner is likely 
to need. 

- An international consultant on CEF was also involved in the process in order 
to oversee the research done for the improvement of the programme, and this 
ensured and confirmed the reliability of the work carried out for the 
implementation of Modular System. 

Modular System 

The modular system consists of 5 levels: Beginner Level (A), Elementary Level 
(B), Pre-Intermediate Level (C), Intermediate Level (D), and Upper- 
Intermediate Level (E). Each level corresponds to one module which takes 7 
weeks. In this respect, students who start their academic year in A Level can 
complete the Preparatory Programme in 5 modules (one academic year) on 
condition that they do not fail in any levels. Each level has its individual aims 
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and objectives which the students have to meet. The aims and objectives of each 
level are also interconnected serving for the purpose of the final outcome of 
Preparatory Programme. The aims for levels A and E are indicated below to 
give an overall idea of what is expected of the students at the beginning and at 
the end of the programme. 



Table 1. Level aims 



Reading 

Level A Level E 

Students can understand very short simple texts, Students can understand complex texts at 
postcards, e-mails, notices and written directions various lengths with a wide vocabulary 
including familiar names, words, and basic range. They can follow the development of 
phrases. an argument; make judgments based on 

writer’s attitude and implied / stated 
opinions. They can interpret graphic sources 
and draw conclusions based on prior 

knowledge and clues from the text. 

Listening 

Level A Level E 

Students can follow very short monologues, Students can follow complex texts 

dialogues, and simple instructions when (monologues, interviews, discussions, 
carefully articulated with long pauses. They can dialogues, lectures). They can understand 
understand very basic phrases and familiar attitudes and viewpoints in complex texts 
words related to areas of most immediate on both concrete and abstract familiar 
priority (very basic personal and family topics. They can listen to lectures on 

infonnation, shopping, local geography, academic topics and take notes for future 

employment, etc.) reference. 

Writing 

Level A Level E 

Students can write simple phrases / sentences Students can write clear, detailed texts on a 
and create simple texts on familiar topics. variety of subjects, synthesizing 

infonnation and arguments from a number 
of sources when necessary. They can repair 
most of their own mistakes and monitor 
their own work with or without the help of a 
pre-prepared checklist. 

Speaking 

Level A Level E 

Students can use phrases and very short Students can express and seek view points, 
sentences to talk about themselves and their exchange relevant information, and 
family. They can ask simple questions to highlight the personal significance of events 
interact about their immediate needs and very and experiences in conversations and 
familiar topics. discussions on topical issues with a degree 

of fluency and spontaneity. They can report 
infonnation and arguments from a number 
of sources. They can give descriptions and 
presentations on a wide range of subjects 
related to their field of interest. 
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As seen in the table, the aims have a simple, General English orientation in the 
early levels, but follow a more academic direction in the upper levels. The 
programme has been designed taking CEF and the faculty programmes into 
consideration. As a result, it is expected that students completing the 
Preparatory Programme will be equipped with General and Academic English, 
both of which are needed in their undergraduate programmes. 



Table 2. Courses / skills and lesson hours in the modular system 



LEVEL A 
(Beginner) 26 hours 


Main Course 


Integrated Skills 


- 


- 


LEVEL B 

(Elementary) 26 hours 


Main Course 


Integrated Skills 


- 


- 


LEVEL C 

(Pre-Intermediate) 26 hours 


Main Course 


Writing 


- 


- 


LEVEL D 

(Intermediate) 28 hours 


Main Course 


Listening & 
Speaking 


Reading 


Writing 


LEVEL E 

(Upper-Intennediate) 28 hours 


Main Course 


Listening & 
Speaking 


Reading 


Writing 



As for assessment, to be eligible to move up to the next level, a student must 
achieve a weighted average of 60. 10% of the final mark is obtained from the 
Assessment Tasks and 90% from the Gateway examination. For each module, 
unsuccessful students have to repeat the module until they pass. In this way, 
students continue to move forward through the levels until they become eligible 
to take the Proficiency Exam. 

Importance of the Study 

The importance of integrating the European dimension of education into the 
existing national curricula is emphasized by the Ministry of Education and 
Board of Education in several studies (Dogan, 2007). It is believed that 
renewing foreign language teaching programmes in accordance with CEF will 
facilitate a clear definition of teaching and learning objectives and materials. 
Thus, it is expected that the results of this study will help the administration 
identify the level of effectiveness of the current English Teaching Programme, 
and also define the strengths and weaknesses of the programme. Another 
significant aspect of this study is that it will contribute to the scant body of 
literature on preparatory school programme evaluation, and also help those 
working in similar settings in both in administration and curriculum 
development to make the necessary adjustments to their programme. 
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METHOD 
Purpose of the Study 

This study aims to describe the process of developing a new teaching 
programme, taking CEF into consideration, at the Preparatory Programme at the 
School of Foreign Fanguages, Izmir University of Economics, and also to 
evaluate the effectiveness of the programme. The study answers the following 
questions: 

Is there a significant difference in students’ perceptions of their own 
English competencies according to gender? 

Is there a significant relationship between students’ proficiency scores 
obtained at the end of the Preparatory Programme and the perceptions of 
their own English competencies in their faculties? 

Is there any significant variation in students’ perceptions of their own 
English competencies in their faculties according to their achieved levels in 
the Preparatory Programme? 

Is there a significant variation in students’ perceptions of their own English 
competence according to the faculties they attend? 

What are the teachers’ perceptions of the students’ competencies in the four 
skills? 

Participants 

Participants in this study were 236 Freshman students, 110 male, 126 female, 
and 48 staff members from 5 different faculties, with teaching experience 
ranging from 2 to over 20 years. All students involved had experienced the 
initial year of the modular system in 2009-2010 academic year. 

Data Collection Method 

After a broad literature study, and careful consideration of objectives, the 
researchers developed a scale as a data collection instrument consisting of the 
abilities and competency level of students. The scale was piloted on 113 
Freshman students from 5 different faculties and 10 faculty members. 
Reliability and factor analysis was conducted. Principle components analysis 
showed that there is one factor with an explained total variance (%59). The 
reliability co-efficiency was found to be 96. A focus group interview with 12 
students was conducted to elicit the students’ general perceptions of their 
competencies in four skills and the learning environment. Interviews with 48 
faculty teachers were carried out to investigate the students’ English language 
competencies in the faculties from the perspective of teachers’. 
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Data Analysis 

Independent t-test was conducted in order to compare the means of male and 
female. One way ANOVA was used in order to compare the means of students’ 
language level and its relation with the proficiency exam; and competency level 
was investigated by using Pearson correlation. 

RESULTS 

This study aims to evaluate the effectiveness of a newly developed English 
language teaching programme based both on CEF and Academic English. The 
results regarding the first question exploring the question of whether there is a 
significant difference in students’ perceptions of their own competencies in 
terms of gender indicate no significant difference between male and female 
students’ perceptions of their own competencies (t(234)=.79, p>.05). It can be 
said that both males and females perceive themselves as equally competent in 
terms of using the English language in their faculties (Table 3.). 

In regard to the second question a significant relationship between students’ 
proficiency scores and perception of their own competencies was found (r (237) 
=.24, p<.01). The results indicate that students’ perceptions regarding their 
competency level in faculties increase relative to the score of the final 
Proficiency exam. 

The third question researched differences in students’ perceptions of language 
competencies in terms of their starting levels at the Preparatory Programme. 
Results showed a significant difference in perception of their own competence 
in terms of levels was found out (F(3,232)=3.7, p<.05). According to the Tukey 
test, the difference between A and C is significant, which indicates that students 
who began the programme at Beginner level perceive themselves less 
competent than those starting at Pre-Intermediate level. The fourth question 
concerned the identification variation in students’ perceptions, if any, of their 
English language competence in terms of four skills, according to their 
particular faculties. The results indicate no significant difference (F (4,224) 
=.28, p>.05), which means that students’ competency level in English has not 
changed according to the faculty (Table 3.) 

The last question examines the teachers’ perceptions about the students’ 
language competencies at faculties. Interviews were made during the teachers’ 
office hours and questions in the scale were discussed with the teachers. The 
points and comments made by the teachers were noted down as follows: 

In general, teachers stated that most students are able to understand the content 
of the English texts, follow lectures in English, and agreed that the education 
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students received at the Preparatory Programme helps them follow their lessons 
in the faculty. However, they stated that students still have difficulty in 
differentiating between main and supporting ideas in texts (9 teachers) 
differentiating important and irrelevant information in texts (13 teachers) 
differentiating facts from opinions in texts (18 teachers) 
paraphrasing and summarizing while writing essays (2 1 teachers) 
taking notes and writing clear and detailed texts (33 teachers) 
reporting ideas in written form by synthesizing information from a number 
of sources (3 1 teachers) 

paraphrasing and summarizing texts orally in English (27 teachers) 

taking part in discussions in English and presenting their ideas orally (21 

teachers) 

A focus group consisting of students revealed that students are able to follow 
lectures in English and communicate with their teachers in English. However, 
they reported difficulty in doing activities which require higher order thinking 
skills, such as differentiating facts from opinions, differentiating important 
information and irrelevant information, synthesizing information from a number 
of sources, and taking an active part in discussions in English. They also 
reported that while they feel comfortable with grammar based and guided 
activities, they feel less competent with activities which require creativity and 
production. 



Table 3. Students’ perception of English competencies according to gender, 
l evel, faculty 





N 


Mean 


Standard Deviation 


GENDER 


Male 


110 


64.79 


13.37 


Female 


126 


63.53 


11.01 


LEVEL 


A Level 


39 


60.12 


12.07 


B Level 


92 


62.70 


12.60 


C Level 


78 


66.88 


9.91 


D Level 


27 


66.74 


14.68 


FACULTY 


Engineering 


44 


64.04 


15.52 


Economics 


67 


64.89 


11.68 


Communication 


30 


62.06 


10.14 


Arts and Sciences 


29 


64.48 


11.46 


Fine Arts and Design 


59 


64.20 


12.03 
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DISCUSSION 

The results of the statistics, interviews with the teachers, and the student focus 
groups indicate that, in general, selecting objectives according to CEF and 
integrating academic skills objectives into these undoubtedly led to a higher 
quality preparatory programme, as both students and faculty teachers find 
language competency sufficient. The only discrepancy is between the students’ 
satisfaction with their own competency level in their faculties and the teachers’ 
belief that students still need to improve the skills and abilities which require 
higher order thinking. 

As the results of the study indicate, the high score of Proficiency exam and the 
level students start at the Preparatory Program are important determining factors 
for students’ perceptions regarding their language competency level in their 
faculties. It is interesting to note that students who started at Pre-Intermediate 
level in the Preparatory programme felt more competent than those in all other 
levels, whether higher or lower. This is to be expected in relation to Beginner 
and Elementary level, but seems surprising in relation to Intermediate and 
Upper Intermediate levels. A possible reason could be that students who start 
the Preparatory programme at Pre-Intermediate level are frequently English- 
medium high school students who are likely to arrive at university with the 
perception of their own competence, originating from their previous academic 
background. The significant difference found between the Pre-Intermediate 
level and the other levels suggests that a careful examination of the objectives is 
needed, especially of the intermediate level, considering difficulty, 
appropriateness, and duration of each module. 

That there is no significant difference among the faculties regarding students’ 
perceptions of competency can be interpreted as the English teaching 
programme, in general, has met the language needs of the faculties. Teachers 
reported during interviews that students can follow lectures in English. 
However, students seem to have been unsuccessful in using some strategies and 
skills closely related with higher order thinking skills. One of the reasons for 
this could be that students come from a culture and an education system that 
tends to rely on memorization rather than the development of thinking skills. 
Thus, integrating academic skills into the CEF specifications is a strategy that is 
likely to have contributed to the prevention of complications which would have 
arisen if higher order thinking skills had been ignored. Many studies on higher 
order thinking skills emphasize the importance of a strong relationship between 
how four language components and thinking skills are taught in English classes. 
Thus, well-written objectives in the teaching programmes do not in themselves 
guarantee the development of student thinking, and, therefore, conscious efforts 
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by teachers should be made in this respect (Perkins, 1993). In addition, teaching 
higher order thinking skills are a long process and this should not only be the 
responsibility of language teachers in the preparatory programme. Course 
teachers at faculties should also design and present materials in a way that 
students could have an opportunity to improve these skills. Therefore, while it is 
essential in English medium universities to integrate academic skills into 
General English regarding the needs of the students, any particular Preparatory 
Programme, even one that blends both the general and the academic, is not 
likely to be sufficient in itself for the success of the students’ undergraduate 
studies. 

The results also revealed differences between the teachers’ and students’ 
perception regarding the objectives. The skills and strategies which we 
integrated into our program, such as paraphrasing and synthesizing, are seen by 
students as only a tool for language competence, while the expectation of the 
faculty teachers is that these skills acquired in the preparatory year should be of 
a sufficient level for effective use throughout their faculty education without 
further instruction. Students’ lack of awareness of the importance of these 
academic skills, and the faculty teachers’ overly high expectations of the 
preparatory programme may be part of the reason for the discrepancy stated 
above. 

CONCLUSION AND IMPLICATIONS 

As indicated in several studies, programme evaluation is a systematic approach 
in identifying strengths and weaknesses of any programme, which leads to 
improvement in teaching programmes for the benefit of both students and 
teachers. The results of this study indicate that, in general, the programme meets 
the expectation of the students by supplying them with both General and 
Academic English. However, the teachers and the focus group students suggest 
that the programme still has room for improvement in some areas, mainly in the 
higher order thinking skills required in academic life. 

Based on the needs put forward by teachers and students, it could be suggested 
that: 

- a series of learner training sessions to raise the awareness of the students 
can be integrated into their orientation program, 

- duration of the modules, particularly for Intermediate and Upper- 
Intermediate can be extended as these levels require more time to focus on 
higher order thinking skills, 

- the order of objectives in Intermediate and Upper-Intermediate Levels can 
be rearranged considering the difficulty level, 
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- some learning strategies and academic skills could be introduced in 
Elementary and Pre- Intermediate level in such a way that students will be 
exposed to them earlier to extend the duration for practice. For example, the 
reading objective “takes notes while reading” is in the objectives of Level 
D. However, this objective can be introduced in Level C. 

Faculty members and the Preparatory Program Curriculum Unit should work in 
cooperation in order to increase awareness of mutual expectations. 
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